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Abstract. We investigate dynamical systems characterized by a time series of 
distinct semi-stable activity patterns, as they are observed in cortical neural activity 
patterns. We propose and discuss a general mechanism allowing for an adiabatic 
continuation between attractor networks and a specific adjoined transient-state 
network, which is strictly dissipative. Dynamical systems with transient states retain 
functionality when their working point is autoregulated - avoiding prolonged periods 
of stasis or drifting into a regime of rapid fluctuations. We show, within a continuous- 
time neural network model, that a single local updating rule for online learning allows 
simultaneously (a) for information storage via unsupervised Hebbian-type learning (b) 
for adaptive regulation of the working point and (c) for the suppression of runaway 
synaptic growth. Simulation results are presented, the spontaneous breaking of time- 
reversal symmetry and link symmetry are discussed. 
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1. Introduction 

Dynamical systems are often classified with respect to their long-time behaviors, which 
might be, e.g., chaotic or regular pj. Of special interest are attractors, cycles and 
limiting cycles, as they determine the fate of all orbits starting within their respective 
basins of attraction. 

Attractor states play a central role in the theory of recurrent neural networks, 
serving the role of memories with the capability to generalize and to reconstruct a 
complete memory from partial initial information [2\. Attractor states in recurrent 
neural networks face however a fundamental functional dichotomy, whenever the network 
is considered as a functional subunit of an encompassing autonomous information 
processing system, viz an autonomous cognitive system p] . The information processing 
comes essentially to a standstill once the trajectory closes in at one of the attractors. 
Restarting the system 'by hand' is a viable option for technical applications of neural 
networks, but not within the context of autonomously operating cognitive systems. 

One obvious way out of this dilemma would be to consider only dynamical systems 
without attractor states, i.e. with a kind of continously ongoing 'fluctuating dynamics', 
as illustrated in Fig. [1], which might possibly be chaotic in the strict sense of dynamical 
system theory. The problem is then, however, the decision-making process. Without 
well defined states, which last for certain minimal periods, the system has no definite 
information-carrying states onto which it could base the generation of its output signals. 
It is interesting to note in this context, that indications for quasi-stationary patterns in 
cortical neural activity have been observed [ll|5l[6]. These quasi-stationary states can 
be analyzed using multivariate time-series analysis, indicating self-organized patterns 
of brain activity [7j. Interestingly, studies of EEG recordings have been interpreted 
in terms of brain states showing aperiodic evolution states going through sequences of 
attractors that on access support the experience of remembering [8]. These findings 
suggest that 'transient state dynamics', as illustrated in Fig. [T|, might be of importance 
for cortical firing patterns. 

It is possible, from the viewpoint of dynamical system theory, to consider transient 
states as well defined periods when the orbit approaches an attractor ruin. With a 
transient attractor, or attractor ruin, we denote here a point in phase space which could 
be turned continously into a stable attractor when tuning certain of the parameters 
entering the evolution equations of the dynamical system. The dynamics slows down 
close to the attractor ruin and well defined transient states emerge within the ensemble 
of dynamical variables. The notion of transient state dynamics is related conceptually to 
chaotic itinerancy [9], a term used to characterize dynamical systems for which chaotic 
high-dimensional orbits stay intermittently close to low-dimensional attractor ruins for 
certain periods. Instability due to dynamic interactions or noise is necessary for the 
appearance of chaotic itinerancy. 

Having argued that transient-state dynamics might be of importance for a wide 
range of real-world dynamical systems, the question is then of how to generate such 
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kind of dynamical behavior in a controllable fashion and in a manner applicable to a 
variety of starting systems. Viz we are interested in neural networks which generate 
transient states dynamics in terms of a meaningful time series of states approaching 
arbitrarily close predefined attractor ruins. 

The approach we will follow here is to start with an original attractor neural network 
and to transform then the set of stable attractors into transient attractors by coupling 
to auxiliary local variables, which we denote 'reservoirs', governed by long time scales. 
We note, that related issues have been investigated in the context of discrete-time, phase 
coupled oscillators [10], for networks aimed at language processing in terms of 'latching 
transitions' [HI [12], and in the context of 'winnerless competitions' [I3[|1H[T5]. Further 
examples of neural networks capable of generating a time-series of subsequent states are 
neural networks with time- dependent asymmetric synaptic strengths [16] or dynamical 
thresholds [I7j. We also note that the occurrence of spontaneous fluctuating dynamics 
has been studied [18], especially in relation to the underlying network geometry [19j. 

An intrinsic task of neural networks is to learn and to adapt to incoming stimuli. 
This implies, for adaptive neural networks, a continuous modification of their dynamical 
properties. The learning process could consequently take the network, if no precautions 
are taken, out of its intended working regime, the regime of transient state dynamics. 
Here we will show that it is possible to formulate local learning rules which keep the 
system in its proper dynamical state by optimizing continously its own working point. 
To be concrete, let us denote with i the average duration of quasi-stable transient states 
and with At the typical time needed for the transition from one quasi-stationary state 
to the next. The dynamical working point can then be defined as the ratio At/i. 

These time scales, i and At, result, for the network of cortical neurons, from 
the properties of the individual neurons, which are essentially time-independent, and 
from the synaptic strengths, which are slow dynamical variables subject to Hebbian- 
type learning [20]. It then follows, that the modifications of the inter-neural synaptic 
strengths have a dual functionality: on one side they are involved in memory storage 
tasks [20], and on the other side they need to retain the working point in the optimal 
regime. Here we show that this dual functionality can be achieved within a generalized 
neural network model. We show that working-point optimization is obtained when 
the Hebbian learning-rule is reformulated as an optimization procedure, resulting in a 
competition among the set of synapses leading to an individual neuron. The resulting 
learning-rule turns out to be closely related to rules found to optimize the memory- 
storage capacity [22j . 

2. Model 

2.1. Clique encoding 

Neural networks with sparse coding, viz with low mean firing rates, have very large 
memory storage capacities [23]. Sparse coding results, in extremis, in a 'one- winner- 
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Figure 1. Illustration of fluctuating (top) and transient state dynamics (bottom). 



take-air configuration, for wliicli a single unit encodes exactly one memory. In this 
limit the storage capacity is, however, reduced again and linearly proportional to the 
network size, as in the original Hopfield model [23]. Here we opt for the intermediate 
case of 'clique encoding'. A clique is, in terms of graph theory, a fully interconnected 
subgraph, as illustrated in Fig. [2] for a 7-site network. Clique encoding corresponds to 
a 'several-winners-take-all' setup. All members of the winning clique mutually excite 
each other while suppressing the activities of all out-of-clique neurons to zero. 

We note, that the number of cliques can be very large. For illustration let us 
consider a random Erdos-Renyi graph with vertices and linking probability p. The 
overall number of cliques containing Z vertices is then statistically given by 



where is the probability of having Z sites of the graph fully interconnected by 

Z{Z — l)/2 edges and where the last term is the probability that every single of the 
N — Z out-of-cliques vertices is not simultaneously connected to all Z sites of the clique. 

Networks with clique encoding are especially well suited for transient state 
dynamics, as we will discuss further below, and are biologically plausible. Extensive 
sensory preprocessing is known to occur in the respective cortical areas of the brain 
[20] . leading to representations of features and objects by individual neurons or small 
cell assemblies. In this framework a site, viz a neural center, of the effective neural 
network considered here corresponds to such a small cell assembly and a clique to a 
stable representation of a memory, by binding together a finite set of features extracted 
by the preprocessing algorithms from the sensory input stream. 

2.2. Continuous time dynamics 

For our study of possible mechanisms of transient state dynamics in the context of 
neural networks we consider i = 1, ...,N artificial neurons with rate encoding Xi{t) and 
continuous time t G [0, oo]. Let us comment shortly on the last point. The majority 
of research in the field of artificial neural networks deals with the case of discrete time 




(1) 




Figure 2. Geometry and simulation results for a small, 7-site network. 

Left: The links with Wij > 0, containing six cliques, (0,1), (0,6), (3,6), (1,2,3) (which 

is highlighted), (4,5,6) and (1,2,4,5). 

Right: As a function of time, the activities Xi{t) (solid lines) and the respective 
reservoirs (pi{t) (dashed lines) for the transient state dynamics (4,5,6) (1,2,3) — > 
(0,6) — * (1,2, 4, 5). For the parameters values see Sect. 12.61 

t = 0,1,2,... [20]. We are however interested, as discussed in the introduction, in 
networks exhibiting autonomously generated dynamical behaviors, as they typically 
occur in the context of complete autonomous cognitive systems. We are therefore 
interested in networks having update rules being compatible with the interaction with 
other components of a cognitive system. Discrete time updating is not suitable in 
this context, since the resulting dynamical characteristics (i) depend on the choice of 
synchronous vs. asynchronous updating and (ii) are strongly influenced when effective 
recurrent loops arise due to the coupling to other components of the autonomous 
cognitive system. We therefore consider and study here a model with continuous time. 

2.3. Neural network model 

We denote the state variables encoding the activity level by Xi{t) and assume them to 
be continuous variables, Xj G [0, 1]. Additionally, we introduce for every site a variable 
ifi{t) G [0, 1], termed 'reservoir', which serves as a fatigue memory facilitating the self 
generated time series of transient states. We consider the following set of differential 
equations: 



Xi 


= (1 - Xi) Q{ri) Ti + Xi Q{-ri) Vi 

N 

= J2 [fw{^i)&{Wij)Wij + Zijf^{(pj) 
J=l 




(2) 


Ti 


Xj 


(3) 
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= r+ (1 - ipi){l - Xi/Xc)Q{Xc - Xi) 
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Figure 3. Left: Illustration of the reservoir functions fz/wi^p): see Eq. [3J of sigmoidal 
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form [21j with respective turning points if, 

Right: Distribution of the synaptic strength for the inhibitory links Zij < 
active excitatory links < Wij < w leading to clique encoding. Note, that w is not 
a strict upper bound, due to the optimization procedure (|lip . The shaded area just 
below zero is related to the inactive Wij, see Eqs. ([5]) and pT|) . 



We now discuss some properties of ([2115]), which are suitably modified Lotka-Volterra 
equations: 

• Normahzation 

Eqs. (EM]) respect the normahzation Xi, (fi G [0, 1], due to the prefactors — Xj), 
(fi and (l — ipi) in Eqs. ([2]) and (jl]), for the respective growth and depletion processes. 
9(r) is the Heaviside-step function: 9(r < 0) = and 0(r > 0) = 1. 

• Synaptic strength 

The synaptic strength is split into an excitatory contribution oc Wij and an 
inhibitory contribution oc Zij, with Wij being the primary variable: The inhibition 
Zij is present only when the link is not excitatory ([5]). We have used z = —1, viz 
\z\ = 1 throughout the manuscript, which then defines the inverse reference unit 
for the time development. 

• Winners-take-all network 

Eqs. (El) and (I3l) describe, in the absence of a coupling to the reservoir via fz/w{'v), 
a competitive winners-take-all neural network with clique encoding. The system 
relaxes towards the next attractor made up of a clique of Z sites {pi, . . . ,pz) 
connected via excitatory links Wp-^p^ > {i,j = 1, .., Z). 

• Reservoir functions 

The reservoir functions fz/w{v) ^ [0, 1] govern the interaction in between the 
activity levels Xi and the reservoir levels (pi. They may be chosen as washed out 
step functions of sigmoidal form [21], with a suitable width and infiection points 
y^i"'/^), see Fig.H 

• Reservoir dynamics 

The reservoir levels of the winning clique depletes slowly, see Eq. (jl]) and Fig. [21 
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and recovers only once the activity level of a given site has dropped below Xc, 
which defines a site to be active when Xi > Xc- The factor (1 — Xi/xc) occurring in 
the reservoir growth process, see the r.h.s. of (HI), serves for a stabilization of the 
transition between two subsequent memory states. When the activity level of a 
given center i drops below Xc, it cannot be reactivated immediately; the reservoir 
cannot fill up again for Xi ~ x^, due to the (1 — Xi/xc) in (jl]). 

• Separation of time scales 

A separation of time scales is obtained when the are much smaller than the 
typical strength of an active excitatory link, i.e. of a typical Wij > 0, leading to 
transient state dynamics. Once the reservoir of a winning clique is depleted, it 
looses, via /^(v^), its ability to suppress other sites and the mutual intra-clique 
excitation is suppressed via fw{f)- 

• Absence of stationary solutions 

There are no stationary solutions with Xj = = (f {i = 1, . . . , A^) for Eqs. ([2]) 
and (jlj), whenever > do not vanish and for any non-trivial coupling functions 

/w.Me[o,i]. 

When decouphng the activities and the reservoir by setting fw/z{v) = 1 obtains 
stable attractors with Xj = 1/0 and = 0/1 for sites belonging/not-belonging to 
the winning clique, compare Fig. HI 

In Fig. [2] the transient state dynamics resulting from Eqs. (^j^ is illustrated. Presented 
in Fig.[2]are data for the autonomous dynamics in the absence of external sensory signals, 
we will discuss the effect of external stimuli further below. We present in Fig. [2] only 
data for a very small network, containing seven sites, which can be easily represented 
graphically. We have also performed extensive simulations for very large networks, 
containing several thousands of sites, and found stable transient state dynamics. 

2.4- Role of the reservoir 

The dynamical system discussed here represents in first place a top-down approach 
to cognitive systems and a one-to-one correspondence with cortical structures is not 
intended. The setput is however inspired by biological analogies and we may identify 
the sites i of the artificial neural network described by Eq. ([2]) not with single neurons, 
but with neural assemblies or neural centers. The reservoir variables ipi{t) could therefore 
be interpreted as effective fatigue processes taking place in continuously active neural 
assemblies, the winning coalisions. 

It has been been proposed [25], that the neural coding used for the binding of 
heterogeneous sensory information in terms of distinct and recognizable objects might 
be temporal in nature. Within this temporal coding hypothesis, which has been 
investigated experimentally [26], neural assemblies fire in phase, synchronous, when 
defining the same object and asynchronous when encoding different objects. There is 
a close relation between objects and memories in general. An intriguing possibility 
is therefore to identify the memories of the transient-state network investigated in 
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the present approach with the synchronous firing neurons of the temporal coding 
theory. The winning coalition is characterized by high reservoir levels which would then 
correspond to the degree of synchronization within the temporal encoding paradigm and 
the reservoir depletion time ~ l/r~ would correspond to the decoherence time of the 
object binding neurons. 

We note, that this analogy can however not be carried to far, since synchronization 
is at its basis a cooperative effect, the reservoir levels describing on the other side single- 
unit properties. In terms of a popular physics phrase one might speak of a 'poor man's' 
approach to synchronization, via coupling to a fatigue variable. 



2.5. Dissipative dynamics 

The reason for the observed numerical and dynamical robustness can be traced back to 
its relaxational nature. For short time scales we can consider the reservoir variables {^Pi} 
to be approximatively constant and the system relaxes into the next clique attractor 
ruin. Once close to a transient attractor, the {xi} are essentially constant, viz close 
to one/zero and the reservoir slowly depletes. The dynamics is robust against noise, 
as fluctuations affect only details of both relaxational processes, but not their overall 
behavior. 

To be precise we note, that the phase space contracts with respect to the reservoir 
variables, namely 

= -T.[^^i'^-^^/^c)e{xc-x,) + T-e{x,-x,)] < 0, Va;, e[0,l], 

i ^ri i 

where we have used dl]). We note that the diagonal contributions to the link matrices 
vanish, za = Q = wu, and therefore dvi/dxi = 0. The phase space contracts consequently 
also with respect to the activities. 



dx. 



< 



where we have used ([2]). The system is therefore strictly dissipative, leading to the 
observed numerically robust behavior. 



2.6. Strict transient state dynamics 

The self-generated transient state dynamics shown in Fig. [2] exhibits well characterized 
plateaus in the Xi (t) , since small values have been used for the depletion and the growth 
rate of the reservoir, F" = 0.005 and = 0.015. The simulations presented in Fig. [2] 
were performed using Wij = 0.12 for all non-zero excitatory interconnections. 

We define a dynamical system to have 'strict transient state dynamics' if there 
exists a set of control parameters allowing to turn the transient states adiabatically into 
stable attractors. Eqs. ([2115]) fulfill this requirements, for F~ ^ the average duration 
t of the steady-stated plateaus observed in Fig. [21 diverges. 
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, {Xi(t),(Pi(t)} 




Figure 4. The attractors of the original network, viz when the coupUng to the 
reservoir is turned off by setting fw/zif) = 1; correspond to Xi — 1/0 and ipi = 0/1 
(i = 1, . . . , N) for member/non- members of the winning clique. A finite coupling to the 
local reservoirs ipi leads to orbit {xi{t),ipi{t)} which are attracted by the attractor ruins 
for short time scales and repelled for long time scales. This is due to a separation of time 
scales, as the time evolution of the reservoirs fi(t) occurs on time scales substantially 
slower than that of the primary dynamical variables Xi(t). 

Alternatively, by selecting appropriate values for and r+, it is possible to 
regulate the 'speed' of the transient state dynamics, an important consideration for 
applications. For a working cognitive system, such as the brain, it is enough that the 
transient states are stable just for a certain minimal period needed to identify the state 
and to act upon it. Anything longer would just be a 'waste of time'. 

2. 7. Universality 

We note that the mechanism for the generation of stable transient state dynamics 
proposed here is universal in the sense that it can be applied to a wide range of dynamical 
systems in a frozen state, i.e. which are determined by attractors and cycles. 

Physically, the mechanism we propose here is to embed the phase space {xj} 
of an attractor network into a larger space, {xi, ipj}, by coupling to additional 
local slow variables ipi. Stable attractors are transformed into attractor ruins since 
the new variables allow the system to escape the basin of the original attractor 
{xi = 1/0, (fj = 0/1} (for in-chque/out-of-chque sites) via local escape processes which 
deplete the respective reservoir levels ipi(t). Note, that the embedding is carried out via 
the reservoir functions fz/w{'^) in Eq. ([3]) and that the reservoir variables keep a slaved 
dynamics (jlj) even when the coupling is turned off by setting fz/w{v) — > 1 in Eq. ([3]). 

This mechanism is illustrated in Fig. HI Locality is an important ingredient for 
this mechanism to work. The trajectories would otherwise not come close to any of the 
attractor ruins again, viz to the original attractors, being repelled by all of them with 
similar strengths and fluctuating dynamics of the kind illustrated in Fig. [1] would result. 

2.8. Cycles and time reversal symmetry 

The systems illustrated in Figs. [2] and [5] are very small and the transient state dynamics 
soon settles into a cycle of attractor ruins, since there are no incoming sensory signals 
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Figure 5. Geometry and simulation results for a cyclic 9-site network with symmetric 
excitatory links Wij = Wji. 

Left: The links with Wi,j > 0, containing six cliques, (0,1), (1,2,3) (which is 
highlighted), (3,4), (4,5,6), (6,7) and (7,8,0). 

Right: As a function of time, the activities Xi{t) for the cyclic transient state dynamics 
(1,2,3) (7,8,0) (4,5,6,) for the parameters values see Sect. \2M Both 

directions (clockwise/anticlockwise) of 'rotation' are dynamically possible and stable, 
the actual direction being determined by the dynamical initial conditions. 

considered in the respective simulations. For networks containing a larger number of 
sites, the number of attractors can be however very large and such the resulting cycle 
length. We performed simulations for a 100-site network, containing 713 clique-encoded 
memories. We found no cyclic behavior even for sequences of transient states containing 
up to 4400 transient states. We note, that the system does not necessarily retrace its 
own trajectory once a given clique is stabilized for a second time, an event which needs 
to occur in any finite system. The reason being, that the distribution of reservoir levels 
is in general different when a given clique is revisited for a second time. 

We note, that time reversal symmetry is 'spontaneously' broken in the sense that 
repetitive transient state dynamics of type 

(clique A) — > (clique B) (clique A) (clique B) — > . . . 

does generally not arise. The reason is simple. Once the first clique is deactivated its 
respective reservoir levels need a certain time to fill up again, compare Fig. [2j Time 
reversal symmetry would be recovered however in the limit r+ ^ F^, i.e. when the 
reservoirs would be refilled much faster than depleted. 

2.9. Reproducible sequence generation 

Animals need to generate sequences of neural activities for a wide range of purposes, e.g. 
for movements or for periodic internal muscle contractions, the heart-beat being a prime 
case. These sequences need to be successions of well defined firing patterns, usable to 
control actuators, viz the muscles. The question then arrises under which condition a 
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dynamical system generates reproducible sequences of well defined activity patterns, i.e. 
controlled time series of transient states [27| [28]. 

There are two points worth noting in this context. 

i. The dynamics described by Eqs. (I2]|l]) works fine for randomly selected link matrices 

Wij which may, or may not change with time passing. In particular one can select 
the cliques specifically in order to induce the generation of a specific succession 
of transient states, an example is presented in Fig. [51 The network is capable, as 
a matter of principle to generate robustly large numbers of different sequences of 
transient states. For geometric arrangements of the networks sites, and of the links 
Wij, one finds waves of transient states sweeping through the system. 

ii. In Sect. [3] we will discuss how appropriate Wij can be learned from training patterns 

presented to the network by an external teacher. We will concentrate in Sect. [3] 
on the training and learning of individual memories, viz of cliques, but suitable 
sequences of training patters could be used also for learning temporal sequences of 
memories. 

3. Autonomous online learning 

An external stimulus, {bf'^^\t)}, influences the activities Xi{t) of the respective neural 
centers. This corresponds to a change of the respective growth rates rj. 



compare Eq. ([3]), where fzi^i) is an appropriate coupling function, depending on the 
local reservoir level When the effect of the external stimulus is strong, namely when 
y.^^(ea;t) g^roug, it wiU in general lead to an activation Xj — > 1 of the respective neural 
center i. A continously active stimulus does not convey new information and should, 
on the other hand, lead to habituation, having a reduced influence on the system. A 
strong, continously present stimulus leads to a prolonged high activity level Xi 1 oi 
the involved neural centers, leading via (HI) to a depletion of the respective reservoir 
levels, on a time scale given by the inverse reservoir depletion rate, l/F". Habituation 
is then mediated by the coupling function fz{fi) in (EI), since fz{^i) becomes very small 
for (fii — > 0, compare Fig. [31 The effect of habituation incorporated in (j6]) therefore 
allows the system to turn its 'attention' to other competing stimuli, with novel stimuli 
having a higher chance to affect the ongoing transient state dynamics. 

We now provide a set of learning rules allowing the system to acquire new patterns 
on the fly, viz during its normal phase of dynamical activity. The alternative, modeling 
networks having distinct periods of learning and of performance, is of widespread use 
for technical applications of neural networks, but is not of interest in our context of 
continuously active cognitive systems. 




(6) 
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3.1. Short- and long term synaptic plasticities 

There are two fundamental considerations for the choice of synaptic plasticities adequate 
for neural networks with transient state dynamics. 

• Learning is a very slow process without a short term memory. Training patterns 
need to be presented to the network over and over again until substantial synaptic 
changes are induced [20]. A short term memory can speed-up the learning process 
substantially, as it stabilizes external patterns and hence gives the system time to 
consolidate long term synaptic plasticity. 

• Systems using sparse coding are based on a strong inhibitory background, 
the average inhibitory link-strength \z\ is substantially larger than the average 
excitatory link strength w, 

\z\ ^ w . 

It is then clear that gradual learning is effective only when it affects dominantly the 
excitatory links: Small changes of large parameters do not lead to new transient 
attractors, nor do they influence the cognitive dynamics substantially. 

It then follows, that it is convenient to split the synaptic plasticities into two parts, 

Wi, = Wi,{t) = w^j{t) + w^j{t) , (7) 

S I L 

where the w^- correspond to the short term and to the long term synaptic plasticities 
respectively. 

3.2. Negative baseline 

Eq. ([5]), Zij = —\z\ Q{—Wij), implies that the inhibitory link strength is either zero or 
— \z\, but is not changed directly during learning, in accordance to above discussion. We 
may therefore consider two kinds of 'excitatory links strengths': 

• Active : An active Wij > is positive and enforces Zij = for the same link, via 
Eq. ©. 

• Inactive : An inactive Wij < is slightly negative, we use Wij = wj^''"''' < as a 
default. It enforces Zij = —\z\ for the same link, via Eq. (JSj) and does not contribute 
to the dynamics, since the excitatory links enter as 6{wij) in ([3]). 

When Wij acquires, during learning, a positive value, the corresponding inhibitory link 
Zij is turned off via Eq. and the excitatory link Wij determines the value of the 
respective term, fw{fi)Q{wij)wij + Zijfz{fj), in Eq. ([3]). We have used a small negative 
baseline of Wj^^^'' = —0.01 throughout the simulations. 

3.3. Short term memory dynamics 

It is reasonable to have a maximal possible value W^"'^^^ for the short term synaptic 
plasticities. We consider therefore the following Hebbian-type learning rule: 

wf^it) = r+ (lyf '^^^ - wfj) e(x. - x,)Q{x, - x,) (8) 
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Figure 6. The time evolution of the short term- memory, for some selected links wfj 
and the network illustrated in Fig. [21 without the link (3,6). The transient states are 
(0,1) (4,5,6) (1,2,3) (3,6) (0,6) (0,1). An external stimulus at sites 
(3) and (6) acts for t G [400,410] with strength = 3.6. 



w^jit) increases rapidly, with rate Tg, when both the pre- and the post-synaptic neural 
centers are active, viz when their respective activities are above Xc- Otherwise it decays 
to zero, with a rate Tg. The coupling functions fz{^) preempt prolonged self-activation 
of the short term memory. When the pre- and the post-synaptic centers are active long 
enough to deplete their respective reservoir levels, the short term memory is shut-off 
via the f,{ip). We have used = 0.1, Tg = 0.0005 and W^""''''^ = 0.02 and = 0.85 
throughout the simulations. 

In Fig. Owe present the time evolution of some selected wfj(t), for a simulation 
using the network illustrated in Fig. [2l The short term memory is activated in three 
cases: 

• When an existing clique, viz a clique encoded in the long term memory w^j, is 
activated, as it is the case of (0,1) for the data presented in Fig. [6], the respective 
intra-clique w^j are also activated. This behavior is a side effect since, for the 
parameter values chosen here, the magnitude of the short term link strengths is 
substantially smaller than those of the long term link strengths. 

• During the transient state dynamics there is a certain overlapping of a currently 
active clique with the subsequent active clique. For this short time span the short 
term plasticities wfj for synapses hnking these two cliques get activated. An example 
is the link (2,4) for the simulation presented in Fig. [61 

• When external stimuli act on two sites not connected by an excitatory long term 
memory link w^j, the short term plasticity wfj makes a qualitative difference. It 
transiently stabilizes the corresponding link and the respective link becomes a new 
chque {i,j) either by itself, or as part of an enlarged and already existing clique. 
An example is the link (3,6) for the simulation presented in Fig. [6l Note however 
that, without subsequent transferal into the long term memory, these new states 
would disappear with a rate once the causing external stimulus is gone. 
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Figure 7. The time evolution of the long term memory, for some selected links ^ 
and the network illustrated in Fig. [51 without the link (3,6). The transient states are 
(0,1) (4,5,6) (1,2,3) (3,6) (0,6) (0,1). An external stimulus at sites 
(3) and (6) acts for t e [400,410] with strength b^"""^^ = 3.6. The stimulus pattern 
(3,6) has been learned by the system, as the w^fi and wg^s turned positive during 
the learning-interval « [400,460]. The learning interval is substantially longer than 
the bare stimulus length due to the activation of the short term memory. The decay 
of certain wfj in the absence of an external stimulus is due to forgetting (|lip . which 
should normally be a very weak effect, but which has been choose here to be a sizeable 
r£ = 0.1, for illustrational purposes 



The last point is the one of central importance, as it allows for the temporal stabilization 
of new patterns present in the sensory input stream. 

3.4- Long term memory dynamics 

Information processing dynamical systems retain their functionalities only when they 
keep their dynamical properties within certain regimes, they need to regulate their 
own working point. For the type of systems discussed here, exhibiting transient-state 
dynamics, the working point is, as discussed in the introduction, defined as the time 
At the system needs for a transition from one quasi- stationary state to the subsequent, 
relative to the length t of the individual quasi- stationary states, which is given by l/F^. 

The cognitive information processing within neural networks occurs on short to 
intermediate time scales. For these processes to work well the mean overall synaptic 
plasticities, viz the average strength of the long term memory links wf'j, needs to be 
regulated homeostatically. The average magnitude of the growth rates r^, see Eq. (jS]), 
determines the time At needed to complete a transition from one winning clique to the 
next transient state. It therefore constitutes a central quantity regulating the working 
point of the system, since i ~ l/F^ is fixed, the reservoir depletion rate F" is not affected 
by learning processes which affect exclusively the inter-neural synaptic strengths. 

The bare growth rates rj(t) are quite strongly time dependent, due to the time- 
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dependence of the postsynaptic reservoirs entering the reservoir function f^{ipi), see Eq. 
([3]). The effective incoming synaptic signal strength 



Wij + Zijf^{(pj) Xj , (9) 

which is independent of the post-synaptic reservoir (fi, is a more convenient local control 
parameter. The working point of the cognitive system is optimal when the effective 
incoming signal is, on the average, of comparable magnitude r^"^^^ for all sites, 

f, r(°f*) . (10) 

The long term memory has two tasks: To extract and encode patterns present in the 
external stimulus, Eq. ([6]), via unsupervised learning and to keep the working point of 
the dynamical system in its desired range. Both tasks can be achieved by a single local 
learning rule. 



:iii 



- d{Wij) 6(xi - Xc) Q{xc - Xj) , (12) 

where Afj = r'-"^*-' — fj. For the numerical simulations we used T^°^^^ = 0.0008, 

now comment on some properties of these evolution 

equations for wfj(t): 
• Hebbian learning 



The learning rule (ITTI) is local and of Hebbian type. Learning occurs only when 
the pre- and the post-synaptic neuron are active, viz when their respective activity 
levels are above the threshold Xc- Weak forgetting, i.e. the decay of seldom used 
links is governed by f[T^ . The function d{wfj) determines the functional dependence 
of forgetting on the actual synaptic strength, we have used d{wfj) = 6{wfj)wfj for 
simplicity. 

Synaptic competition 

When the effective incoming signal fj is weak/strong, relative to the optimal value 
j.(opt) ^ the active links are reinforced/weakened, with 14^^™"-* being the minimal 
value for the Wij. The baseline wj^^^^ is slightly negative, compare Figs. [3] and [71 
The Hebbian-type learning then takes place in the form of a temporal competition 
among incoming synapses - frequently active incoming links will gain strength, on 
the average, on the expense of rarely used links. 

Fast learning of new patterns 

In Fig. [7| the time evolution of some selected wf"j is presented. A simple input 
pattern is learned by the network. In this simulation the learning parameter r^°^*'' 
has been set to a quite large value such that the learning occurs in one step (fast 
learning) . 

Suppression of runaway synaptic growth 

When a neural network is exposed repeatedly to the same, or to similar external 
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stimuli, unsupervised learning generally then leads to uncontrolled growth of the 
involved synaptic strengths. This phenomena, termed 'runaway synaptic growth' 
can also occur in networks with continuous self-generated activities, when similar 
activity patterns are auto-generated over and over again. Both kinds of synaptic 
runaway-growth is suppressed by the proposed link-dynamics ( ITTl) . 
• Negative baseline 

Note that Wij = w^j + w^^ enters the evolution equation ([3]) as 6{wij). We can 
therefore distinguish between active {wij > 0) and inactive {wij < 0) configuration, 
compare Fig. [31 The negative baseline wj^^'^^ < entering ffTTl) then allows for the 
removal of positive links and provides a barrier against small random fluctuations, 
compare Sect. 13. 2[ 

During a transient state we have Xj 1 for all vertices belonging to the winning coalition 
and Xj — > for all out-of-clique sites, leading to 

fi ~ ^ Wij, j G active sites , 

j 

compare Eq. Qj. The working-point optimization rule ffTOl) . fj — > r^°P^'^ is therefore 
equivalent to a local normalization condition enforcing the sum of active incoming link- 
strengths to be constant, i.e. site independent. This rule is closely related to a mechanism 
of self regulation of the average firing rate of cortical neurons proposed by Bienenstock, 
Cooper and Munro [30] . 

3.5. Online learning 

The neural network we consider here is continously active, independently of whether 
there is sensory input via Eq. ([6]) or not. The reason being, that the evolution equations 
([2]) and (jl]) generate a never ending time series of transient states. It is a central 
assumption of the present study, that continuous and self-generated neural activity is a 
condition sine qua no for modeling overall brain activity or for developing autonomous 
cognitive systems 

The evolution equations for the synaptic plasticities, namely (IHl) for the short-term 
memory and ffTTl) for the long-term memory are part of the dynamical system, viz they 
determine the time evolution of wfj{t) and of w^j{t) at all times, irrespectively of whether 
external stimuli are presented to the network via or not. The evolution equations 
for the synaptic plasticities need therefore to fulfill, quite in general for a continously 
active neural network, two conditions: 

(a) Under training conditions, namely when input patterns are presented to the system 
via (Q, the system should be able to modify the synaptic link strength accordingly, 
such that the training patterns are stored in the form of new memories, viz cliques 
representing attractor ruins and leading to quasistationary states. 

(b) In the absence of input the ongoing transient-state dynamics will lead constantly 
to synaptic modifications, via (IHl) and ( ITTl) . Theses modification may not induce 
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qualitative changes, such as the the autonomous destruction of existing memories 
or the spontaneous generation of spurious new memories. New memories should be 
acquired exclusively via training by external stimuli. 

In the following section we will present simulations in order to investigate these points. 
We find that the evolution equations formulated in this study conform with both 
conditions (a) and (b) above, due to the optimization principle for the long-term synaptic 
plasticities in Eq. ffTTl) . 

4. Simulations 

We have performed extensive simulations of the dynamics of the network with ongoing 
learning, for systems with up to several thousands of sites. We found that the dynamics 
remains long-term stable even in the presence of continuous online learning governed 
by Eqs. (IHl) and ( JTTll . exhibiting semi-regular sequences of winning coalition, as shown 
in Figs. [21 The working point is regulated adaptively and no prolonged periods of 
stasis or trapped states were observed in the simulations, neither did periods of rapid 
or uncontrolled oscillations occur. 

Any system with a finite number of sites and a finite number of cliques settles 
in the end, in the absence of external signals, into a cyclic series of transient states. 
Preliminary investigations of systems with ^ 20 — 100 resulted in cycles spanning on 
the average a finite fraction of the set of all cliques encoded by the network. This is a 
notable result, since the overall number of cliques stored in the network can easily be 
orders of magnitudes larger than the number of sites N itself, compare Eq. ([T]). Detailed 
studies of the cyclic behavior for autonomous networks will be presented elsewhere. 

4.I. Learning of new memories 

Training patterns {pi,--,Pz} presented to the system externally via ^ ri + 
fw{'^i)i>f^^\t)i for i G {pii ■■iPz}-, are learned by the network via the activation of 
the short term memory for the corresponding intra-pattern links. In Figs. [6] and [7] we 
present a case study. The ongoing internal transient state dynamics is interrupted at 
time t = 400 by an external signal which activates the short term memory, see Fig. [61 
Note that the short term memory is activated both by external stimuli and internally 

Table 1. Learning results for systems with N sites and Nunks excitatory links and 
N2,..,Nq cliques containing 2, ..,6 sites. Ntot is the total number memories to be 
learned. Ni and Npar denote the number of memories learned completely/partially. 
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N2 


ATg 
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A^ 
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20 
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Figure 8. The link-asymmetry wfj — wj< for the positive w^- for a 100-site network 
with 713 cliques at time t — 5 ■ 10^, corresponding to circa 4500 transient states. 
Left: After learning from scratch. Training was finished at t « 5 • lO**. 
Right: Starting with Wij 0.12 for all links belonging to one or more cliques. 



whenever a given link becomes active, i.e. when both pre- and post-synaptic sites are 
active coinstantaneous. The internal activation does however not lead to the internal 
generation of spurious memories, since internally activated links belong anyhow to one 
or more already existing cliques. 

In Fig. [7] we present the time development of the respective long term synaptic 
modifications, wjlj{t). The parameters for learning chosen here allow for fast learning, 
the pattern corresponding to the external signal, retained temporarily in the short term 
memory, is memorized in one step, viz the corresponding w^Q{t) becomes positive before 
the transition to the next clique takes place. For practical applications smaller learning 
rates might be more suitable, as they allow to avoid learning of spurious signals generated 
by environmental noise. 

In Table [1] we present the results for the learning of two networks with = 20 
and = 100 from scratch. The initial networks contained only two connected cliques, 
in order to allow for a non-trivial initial transient state dynamics, all other links where 
inhibitory. Learning by training and storage of the externally presented patterns, using 
the same parameters as for Figs. [6] and U\ is nearly perfect. The learning rate can be 
chosen over a very wide range, as we tested. Here the training phase was completed, for 
the 100-site network, by t = 5 ■ 10^. Coming back to the discussion in section 13.51 we 
then conclude that the network fulfills the there formulated condition (a), being able to 
store efficiently training patterns as attractor ruins in the form of cliques. 

4-2. Link asymmetry 

We note that the Hebbian learning via the working-point optimization, Eq. ( fTTil . leads 
to the spontaneous generation of asymmetries in the link matrices, viz to w[j ^ w^^, 
since the synaptic plasticity depends on the postsynaptic growth rates. 

In Fig. [8] we present, for two simulations, the distribution of the link-asymmetry 
w^j — for all positive w^j, for the 100-site network of Tabled! at time t = 5 - 10^. The 



Transient state dynamics 



19 



distributions shown in Fig. [Dare particular realizations of steady-state distributions, viz 
they did not change appreciably for wide ranges of total simulation times. 

(i) In the first simulation the network had been learned from scratch. The set of 713 
training patterns were presented to the network for t G [0,5 ■ 10^]. After that, for 
t G [5 ■ 10^, 5 ■ 10^] the system evolved freely. A total of 3958 transient states had 
been generated at time t = 5 ■ 10^, but the system had nevertheless not yet settled 
into a cycle of transient states, due to the ongoing synaptic optimization, Eqs. ([8]) 
and (iTTl) . 661 cliques remained at t = 5 ■ 10^, as the link competition had led to 
the suppression of some seldom used links. 

(ii) In the second simulation, uniform and symmetric starting excitatory links wjj — >■ 
0.12 had been set by hand at t = 0, for all intra-clique links. The same N = 100 
network as in (i) was used and the simulation ran in the absence of external 
stimuli. All 713 cliques were still present at t = 5 • 10^, despite the substantial 
reorganization of the link-strength distribution, from the initial uniform to the 
stationary distribution shown in Fig. [HI A total of 4123 transient states had been 
generated in the course of the simulation, without the system entering into a cycle. 

For both simulations all evolution equations, namely and (jll) for the activities and 
reservoir levels, as well as ([8]) for the short-term memory and (fTT!) and (fT2l) for the 
long term memory determined the dynamics for all times t G [0, 5 ■ 10^]. The difference 
between (i) and (ii) being the way the memories are determined, via training by external 
stimuli, Eq. as in (i) or by hand as in (ii). 

Comparing the two link distributions shown in Fig. [8], we note the overall similarity, 
a consequence of the continuously acting working-point optimization. The main 
differences turn-up for small link strengths, since these two simulations started from 
opposite extremes (vanishing/strong initial excitatory links). The details of the link 
distribution shown in Fig. [8] depend sensitively on the parameters. For the results show 
in Fig. [8] we used for illustrational purposes = 0.1, which is a very big value for 
a parameter regulating weak forgetting. We also performed simulations with F^ = 0, 
the other extreme, and found that the link-asymmetry distribution was somewhat more 
scattered. 

Coming back to the discussion in section 13.51 we then conclude that the network 
fulfills the there formulated condition (b), since essentially no memories acquired during 
the training state were destroyed, or spurious new memories spontaneously creasted, 
during the subsequent free evolution. 

5. Conclusions 

We have investigated a series of issues regarding neural networks with autonomously 
generated transient state dynamics. We have presented a general method allowing to 
transform an initial attractor network into a network capable of generating an infinite 
time series of transient states. The resulting dynamical system has strictly contracting 
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phase space, with a one-to-one adiabatic correspondence between the transient states 
and the attractors of the original network. 

We then have discussed the problem of homeostasis, namely the need for the system 
to regulate its own working point adaptively. We formulated a simple learning rule for 
unsupervised local Hebbian-type learning, which solves the homeostasis problem. We 
note here, that this rule, Eq. ffTTl) is similar to learning rules shown to optimize the 
overall storage capacity for discrete-time neural networks |22j . 

We have studied a continuous time neural network model using clique encoding 
and showed that this model is very suitable for studying transient state dynamics in 
conjunction with ongoing learning-on-the-fly for a wide range of learning conditions. 
Both fast and slow online learning of new memories is compatible with the transient 
state dynamics self-generated by the network. 

Finally we turn to the interpretation of the transient state dynamics. Examination 
of a typical time series of subsequently activated cliques, as the one shown in Fig. [21 
reveals that the sequence of cliques is not random. Every single clique is connected to 
its predecessor via excitatory links, they are said to be 'associatively' connected [29] . 
The sequence of subsequently active cliques can therefore be viewed, cum grano salis, 
as an 'associative thought process' [29]. The possible use of such processes for cognitive 
information processing needs, however, yet to be investigated. 
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